Toward More Comprehensive Evaluations of 3D Immersive Sketching, Drawing, and Painting.

To understand current practice and explore the potential for more comprehensive evaluations of 3D immersive sketching, drawing, and painting, we present a survey of evaluation methodologies used in existing 3D sketching research, a breakdown and discussion of important phases (sub-tasks) in the 3D sketching process, and a framework that suggests how these factors can inform evaluation strategies in future 3D sketching research. Existing evaluations identified in the survey are organized and discussed within three high-level categories: 1) evaluating the 3D sketching activity, 2) evaluating 3D sketching tools, and 3) evaluating 3D sketching artifacts. The new framework suggests targeting evaluations to one or more of these categories and identifying relevant user populations. In addition, building upon the discussion of the different phases of the 3D sketching process, the framework suggests to evaluate relevant sketching tasks, which may range from low-level perception and hand movements to high-level conceptual design. Finally, we discuss limitations and challenges that arise when evaluating 3D sketching, including a lack of standardization of evaluation methods and multiple, potentially conflicting, ways to evaluate the same task and user interface usability; we also identify opportunities for more holistic evaluations. We hope the results can contribute to accelerating research in this domain and, ultimately, broad adoption of immersive sketching systems.

Inward VR: Toward a Qualitative Method for Investigating Interoceptive Awareness in VR.

Immersive virtual reality (VR) technologies can produce powerful illusions of being in another place or inhabiting another body, and theories of presence and embodiment provide valuable guidance to designers of VR applications that use these illusions to "take us elsewhere." However, an increasingly common design goal for VR experiences is to develop a deeper awareness of the internal landscape of one's own body (i.e., interoceptive awareness); here, design guidelines and evaluative techniques are less clear. To address this, we present a methodology, including a reusable codebook, for adapting the five dimensions of the Multidimensional Assessment of Interoceptive Awareness (MAIA) conceptual framework to explore interoceptive awareness in VR experiences via qualitative interviews. We report results from a first exploratory study (n=21) applying this method to understand the interoceptive experiences of users in a VR environment. The environment includes a guided body scan exercise with a motion-tracked avatar visible in a virtual mirror and an interactive visualization of a biometric signal detected via a heartbeat sensor. The results provide new insights on how this example VR experience might be refined to better support interoceptive awareness and how the methodology might continue to be refined for understanding other "inward-facing" VR experiences.

The Ball is in Our Court: Conducting Visualization Research With Sports Experts.

Most sports visualizations rely on a combination of spatial, highly temporal, and user-centric data, making sports a challenging target for visualization. Emerging technologies, such as augmented and mixed reality (AR/XR), have brought exciting opportunities along with new challenges for sports visualization. We share our experience working with sports domain experts and present lessons learned from conducting visualization research in SportsXR. In our previous work, we have targeted different types of users in sports, including athletes, game analysts, and fans. Each user group has unique design constraints and requirements, such as obtaining real-time visual feedback in training, automating the low-level video analysis workflow, or personalizing embedded visualizations for live game data analysis. In this article, we synthesize our best practices and pitfalls we identified while working on SportsXR. We highlight lessons learned in working with sports domain experts in designing and evaluating sports visualizations and in working with emerging AR/XR technologies. We envision that sports visualization research will benefit the larger visualization community through its unique challenges and opportunities for immersive and situated analytics.

V-Mail: 3D-Enabled Correspondence about Spatial Data on (Almost) All Your Devices.

We present V-Mail, a framework of cross-platform applications, interactive techniques, and communication protocols for improved multi-person correspondence about spatial 3D datasets. Inspired by the daily use of e-mail, V-Mail seeks to enable a similar style of rapid, multi-person communication accessible on any device; however, it aims to do this in the new context of spatial 3D communication, where limited access to 3D graphics hardware typically prevents such communication. The approach integrates visual data storytelling with data exploration, spatial annotations, and animated transitions. V-Mail "data stories" are exported in a standard video file format to establish a common baseline level of access on (almost) any device. The V-Mail framework also includes a series of complementary client applications and plugins that enable different degrees of story co-authoring and data exploration, adjusted automatically to match the capabilities of various devices. A lightweight, phone-based V-Mail app makes it possible to annotate data by adding captions to the video. These spatial annotations are then immediately accessible to team members running high-end 3D graphics visualization systems that also include a V-Mail client, implemented as a plugin. Results and evaluation from applying V-Mail to assist communication within an interdisciplinary science team studying Antarctic ice sheets confirm the utility of the asynchronous, cross-platform collaborative framework while also highlighting some current limitations and opportunities for future work.

Designing Technology for Sociotechnical Problems: Challenges and Considerations.

Designing technology for sociotechnical problems is challenging due to the heterogeneity of stakeholders' needs, the diversity among their values and perspectives, and the disparity in their technical skills. Careful considerations are needed to ensure that data collection is inclusive and representative of the target populations. It is also important to employ data analysis methods that are compatible with users' technical skills and are capable of drawing a representative picture of people's values, priorities, and needs. However, current technical solutions often fail to meet these critical requirements. In this article, we present a set of empirically-driven design considerations for building technological interventions to address sociotechnical issues. We then discuss open challenges and tradeoffs around privacy, ethics, bias, uncertainty, and trust. We conclude with a call to action for researchers to advance the domain knowledge and improve our technological arsenal for addressing sociotechnical problems.

How to Ask What to Say?: Strategies for Evaluating Natural Language Interfaces for Data Visualization.

In this article, we discuss challenges and strategies for evaluating natural language interfaces (NLIs) for data visualization. Through an examination of prior studies and reflecting on own experiences in evaluating visualization NLIs, we highlight benefits and considerations of three task framing strategies: Jeopardy-style facts, open-ended tasks, and target replication tasks. We hope the discussions in this article can guide future researchers working on visualization NLIs and help them avoid common challenges and pitfalls when evaluating these systems. Finally, to motivate future research, we highlight topics that call for further investigation including development of new evaluation metrics, and considering the type of natural language input (spoken versus typed), among others.

Close Reading for Visualization Evaluation.

Visualizations produced by collaborations between artists, scientists, and visualization experts lay claim to being not only more effective in delivering information but also more effective in their abilities to elicit qualities like human connection. However, as prior work in the visualization community has demonstrated, it is difficult to evaluate these claims because characteristics associated with human connection are not easily measured quantitatively. In this Visualization Viewpoints piece, we address this problem in the context of our work to develop methods of evaluating visualizations created by Sculpting Visualization, a multidisciplinary project that incorporates art and design theory and practice into the process of scientific visualization. We present the design and results of a study in which we used close reading, a formal methodology used by humanities scholars, as a way to test reactions and analyses from evaluation participants related to an image created using Sculpting Visualization. In addition to specific suggestions about how to improve future iterations of the visualization, we discuss key findings of the evaluation related to contextual information, visual perspective, and associations that individual viewers brought to bear on their experience with the visualization.

Under Water to Outer Space: Augmented Reality for Astronauts and Beyond.

Augmented reality (AR) has the potential to help astronauts execute procedures in a quicker, more intuitive, and safer way. A key part of realizing these benefits has been the use of an undersea research facility-the Aquarius-that acts as an analog to the International Space Station to a certain extent. In a June 2019 mission, the Aquarius crew successfully executed a complex procedure taking place across four different task areas by using an AR application called ProtoSpace developed at the Jet Propulsion Laboratory. In this article, we share the detailed results of the study, lessons learned, and future work needed to further enable the enhancement of procedure execution through augmented reality.

Artifact-Based Rendering: Harnessing Natural and Traditional Visual Media for More Expressive and Engaging 3D Visualizations.

We introduce Artifact-Based Rendering (ABR), a framework of tools, algorithms, and processes that makes it possible to produce real, data-driven 3D scientific visualizations with a visual language derived entirely from colors, lines, textures, and forms created using traditional physical media or found in nature. A theory and process for ABR is presented to address three current needs: (i) designing better visualizations by making it possible for non-programmers to rapidly design and critique many alternative data-to-visual mappings; (ii) expanding the visual vocabulary used in scientific visualizations to depict increasingly complex multivariate data; (iii) bringing a more engaging, natural, and human-relatable handcrafted aesthetic to data visualization. New tools and algorithms to support ABR include front-end applets for constructing artifact-based colormaps, optimizing 3D scanned meshes for use in data visualization, and synthesizing textures from artifacts. These are complemented by an interactive rendering engine with custom algorithms and interfaces that demonstrate multiple new visual styles for depicting point, line, surface, and volume data. A within-the-research-team design study provides early evidence of the shift in visualization design processes that ABR is believed to enable when compared to traditional scientific visualization systems. Qualitative user feedback on applications to climate science and brain imaging support the utility of ABR for scientific discovery and public communication.

The Coupled Kinematics of Scapulothoracic Upward Rotation.

BACKGROUND: Scapulothoracic upward rotation (UR) is an important shoulder complex motion allowing for a larger functional work space and improved glenohumeral muscle function. However, the kinematic mechanisms producing scapulothoracic UR remain unclear, limiting the understanding of normal and abnormal shoulder movements. OBJECTIVE: The objective of this study was to identify the coupling relationships through which sternoclavicular and acromioclavicular joint motions contribute to scapulothoracic UR. DESIGN: This was a cross-sectional observational study. METHODS: Sixty participants were enrolled in this study; 30 had current shoulder pain, and 30 had no history of shoulder symptoms. Shoulder complex kinematics were quantified using single-plane fluoroscopy and 2D/3D shape matching and were described as finite helical displacements for 30-degree phases of humerothoracic elevation (30 degrees-60 degrees, 60 degrees-90 degrees, and 90 degrees-120 degrees). A coupling function was derived to estimate scapulothoracic UR from its component motions of acromioclavicular UR, sternoclavicular posterior rotation, and sternoclavicular elevation as a function of acromioclavicular internal rotation. The proportional contributions of each of the component motions were also calculated and compared between phases of humerothoracic elevation and groups. RESULTS: Scapulothoracic UR displacement could be effectively predicted using the derived coupling function. During the 30- to 60-degree humerothoracic elevation phase, acromioclavicular UR accounted for 84.2% of scapulothoracic UR, whereas sternoclavicular posterior rotation and elevation each accounted for < 10%. During later phases, acromioclavicular UR and sternoclavicular posterior rotation each accounted for 32% to 42%, whereas sternoclavicular elevation accounted for < 11%. LIMITATIONS: Error due to the tracking of sternoclavicular posterior rotation may have resulted in an underprediction of its proportional contribution and an overprediction of the proportional contribution of acromioclavicular UR. CONCLUSIONS: Acromioclavicular UR and sternoclavicular posterior rotation are the predominant component motions of scapulothoracic UR. More research is needed to investigate how these coupling relationships are affected by muscle function and influenced by scapular dyskinesis.

Modeling distributed forces within cell adhesions of varying size on continuous substrates.

Cell migration and traction are essential to many biological phenomena, and one of their key features is sensitivity to substrate stiffness, which biophysical models, such as the motor-clutch model and the cell migration simulator can predict and explain. However, these models have not accounted for the finite size of adhesions, the spatial distribution of forces within adhesions. Here, we derive an expression that relates varying adhesion radius ( R) and spatial distribution of force within an adhesion (described by s) to the effective substrate stiffness ( κsub ), as a function of the Young's modulus of the substrate ( E Y ), which yields the relation, κsub=RsEY , for two-dimensional cell cultures. Experimentally, we found that a cone-shaped force distribution ( s = 1.05) can describe the observed displacements of hydrogels deformed by adherent U251 glioma cells. Also, we found that the experimentally observed adhesion radius increases linearly with the cell protrusion force, consistent with the predictions of the motor-clutch model with spatially distributed clutches. We also found that, theoretically, the influence of one protrusion on another through a continuous elastic environment is negligible. Overall, we conclude cells can potentially control their own interpretation of the mechanics of the environment by controlling adhesion size and spatial distribution of forces within an adhesion.

An Emotional Response to the Value of Visualization.

When assessing the value of visualizations, researchers traditionally focus on efficiency, comprehension, or insight. However, analyzing successful data physicalizations leads to a deep appreciation for hedonic qualities. Informed by the role of emotion in psychology, art, design, marketing, and HCI, we argue for an expanded definition of value, applicable to all forms of data visualization.

Bento Box: An Interactive and Zoomable Small Multiples Technique for Visualizing 4D Simulation Ensembles in Virtual Reality.

We present Bento Box, a virtual reality data visualization technique and bimanual 3D user interface for exploratory analysis of 4D data ensembles. Bento Box helps scientists and engineers make detailed comparative judgments about multiple time-varying data instances that make up a data ensemble (e.g., a group of 10 parameterized simulation runs). The approach is to present an organized set of complementary volume visualizations juxtaposed in a grid arrangement, where each column visualizes a single data instance and each row provides a new view of the volume from a different perspective and/or scale. A novel bimanual interface enables users to select a sub-volume of interest to create a new row on-the-fly, scrub through time, and quickly navigate through the resulting virtual "bento box." The technique is evaluated through a real-world case study, supporting a team of medical device engineers and computational scientists using in-silico testing (supercomputer simulations) to redesign cardiac leads. The engineers confirmed hypotheses and developed new insights using a Bento Box visualization. An evaluation of the technical performance demonstrates that the proposed combination of data sampling strategies and clipped volume rendering is successful in displaying a juxtaposed visualization of fluid-structure-interaction simulation data (39 GB of raw data) at interactive VR frame rates.

Drag and Track: A Direct Manipulation Interface for Contextualizing Data Instances within a Continuous Parameter Space.

We present a direct manipulation technique that allows material scientists to interactively highlight relevant parameterized simulation instances located in dimensionally reduced spaces, enabling a user-defined understanding of a continuous parameter space. Our goals are two-fold: first, to build a user-directed intuition of dimensionally reduced data, and second, to provide a mechanism for creatively exploring parameter relationships in parameterized simulation sets, called ensembles. We start by visualizing ensemble data instances in dimensionally reduced scatter plots. To understand these abstract views, we employ user-defined virtual data instances that, through direct manipulation, search an ensemble for similar instances. Users can create multiple of these direct manipulation queries to visually annotate the spaces with sets of highlighted ensemble data instances. User-defined goals are therefore translated into custom illustrations that are projected onto the dimensionally reduced spaces. Combined forward and inverse searches of the parameter space follow naturally allowing for continuous parameter space prediction and visual query comparison in the context of an ensemble. The potential for this visualization technique is confirmed via expert user feedback for a shock physics application and synthetic model analysis.

Maternal-Fetal Medicine Workforce Survey: Are We Ready for Regionalized Levels of Maternal Care?

OBJECTIVE: To conduct a survey of the members of the Society for Maternal-Fetal Medicine (SMFM) to determine the practice patterns of maternal-fetal medicine (MFM) subspecialists in the United States and to estimate the likelihood that our work force is sufficient to support the proposed MFM staffing requirements for level III and IV maternity centers. STUDY DESIGN: All regular SMFM members in the United States were invited to answer a 26 question survey by email. The survey queried demographic characteristics, practice type, night call arrangements, and whether the respondent's hospital was currently equipped with the services and personnel described in the requirements for level III or IV centers. RESULTS: Of the MFM specialists working full time in what would be considered a level III or IV maternity center, only 47.5% took in-house call. Of those taking only call from home or back-up call, the majority reported that during call hours, MFM antepartum and laboring patients are cared for by generalist obstetrician gynecologists; only 6.4% work with MFM hospitalists. Respondents from level III or IV centers also reported that many of their centers did not meet the criteria for nursing support, anesthesia support, or intensive care services. CONCLUSION: These data, if confirmed, indicate that work needs to be done to upgrade services and achieve appropriate staffing to meet the proposed level III and IV criteria for maternity care.

Microstructure Imaging of Crossing (MIX) White Matter Fibers from diffusion MRI.

Diffusion MRI (dMRI) reveals microstructural features of the brain white matter by quantifying the anisotropic diffusion of water molecules within axonal bundles. Yet, identifying features such as axonal orientation dispersion, density, diameter, etc., in complex white matter fiber configurations (e.g. crossings) has proved challenging. Besides optimized data acquisition and advanced biophysical models, computational procedures to fit such models to the data are critical. However, these procedures have been largely overlooked by the dMRI microstructure community and new, more versatile, approaches are needed to solve complex biophysical model fitting problems. Existing methods are limited to models assuming single fiber orientation, relevant to limited brain areas like the corpus callosum, or multiple orientations but without the ability to extract detailed microstructural features. Here, we introduce a new and versatile optimization technique (MIX), which enables microstructure imaging of crossing white matter fibers. We provide a MATLAB implementation of MIX, and demonstrate its applicability to general microstructure models in fiber crossings using synthetic as well as ex-vivo and in-vivo brain data.

Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR.

Three-dimensional modeling has long been regarded as an ideal application for virtual reality (VR), but current VR-based 3D modeling tools suffer from two problems that limit creativity and applicability: (1) the lack of control for freehand modeling, and (2) the difficulty of starting from scratch. To address these challenges, we present Lift-Off, an immersive 3D interface for creating complex models with a controlled, handcrafted style. Artists start outside of VR with 2D sketches, which are then imported and positioned in VR. Then, using a VR interface built on top of image processing algorithms, 2D curves within the sketches are selected interactively and "lifted" into space to create a 3D scaffolding for the model. Finally, artists sweep surfaces along these curves to create 3D models. Evaluations are presented for both long-term users and for novices who each created a 3D sailboat model from the same starting sketch. Qualitative results are positive, with the visual style of the resulting models of animals and other organic subjects as well as architectural models matching what is possible with traditional fine art media. In addition, quantitative data from logging features built into the software are used to characterize typical tool use and suggest areas for further refinement of the interface.

Visualization-by-Sketching: An Artist's Interface for Creating Multivariate Time-Varying Data Visualizations.

We present Visualization-by-Sketching, a direct-manipulation user interface for designing new data visualizations. The goals are twofold: First, make the process of creating real, animated, data-driven visualizations of complex information more accessible to artists, graphic designers, and other visual experts with traditional, non-technical training. Second, support and enhance the role of human creativity in visualization design, enabling visual experimentation and workflows similar to what is possible with traditional artistic media. The approach is to conceive of visualization design as a combination of processes that are already closely linked with visual creativity: sketching, digital painting, image editing, and reacting to exemplars. Rather than studying and tweaking low-level algorithms and their parameters, designers create new visualizations by painting directly on top of a digital data canvas, sketching data glyphs, and arranging and blending together multiple layers of animated 2D graphics. This requires new algorithms and techniques to interpret painterly user input relative to data "under" the canvas, balance artistic freedom with the need to produce accurate data visualizations, and interactively explore large (e.g., terabyte-sized) multivariate datasets. Results demonstrate a variety of multivariate data visualization techniques can be rapidly recreated using the interface. More importantly, results and feedback from artists support the potential for interfaces in this style to attract new, creative users to the challenging task of designing more effective data visualizations and to help these users stay "in the creative zone" as they work.

Trend-Centric Motion Visualization: Designing and Applying a New Strategy for Analyzing Scientific Motion Collections.

In biomechanics studies, researchers collect, via experiments or simulations, datasets with hundreds or thousands of trials, each describing the same type of motion (e.g., a neck flexion-extension exercise) but under different conditions (e.g., different patients, different disease states, pre- and post-treatment). Analyzing similarities and differences across all of the trials in these collections is a major challenge. Visualizing a single trial at a time does not work, and the typical alternative of juxtaposing multiple trials in a single visual display leads to complex, difficult-to-interpret visualizations. We address this problem via a new strategy that organizes the analysis around motion trends rather than trials. This new strategy matches the cognitive approach that scientists would like to take when analyzing motion collections. We introduce several technical innovations making trend-centric motion visualization possible. First, an algorithm detects a motion collection's trends via time-dependent clustering. Second, a 2D graphical technique visualizes how trials leave and join trends. Third, a 3D graphical technique, using a median 3D motion plus a visual variance indicator, visualizes the biomechanics of the set of trials within each trend. These innovations are combined to create an interactive exploratory visualization tool, which we designed through an iterative process in collaboration with both domain scientists and a traditionally-trained graphic designer. We report on insights generated during this design process and demonstrate the tool's effectiveness via a validation study with synthetic data and feedback from expert musculoskeletal biomechanics researchers who used the tool to analyze the effects of disc degeneration on human spinal kinematics.

Design by dragging: an interface for creative forward and inverse design with simulation ensembles.

We present an interface for exploring large design spaces as encountered in simulation-based engineering, design of visual effects, and other tasks that require tuning parameters of computationally-intensive simulations and visually evaluating results. The goal is to enable a style of design with simulations that feels as-direct-as-possible so users can concentrate on creative design tasks. The approach integrates forward design via direct manipulation of simulation inputs (e.g., geometric properties, applied forces) in the same visual space with inverse design via 'tugging' and reshaping simulation outputs (e.g., scalar fields from finite element analysis (FEA) or computational fluid dynamics (CFD)). The interface includes algorithms for interpreting the intent of users' drag operations relative to parameterized models, morphing arbitrary scalar fields output from FEA and CFD simulations, and in-place interactive ensemble visualization. The inverse design strategy can be extended to use multi-touch input in combination with an as-rigid-as-possible shape manipulation to support rich visual queries. The potential of this new design approach is confirmed via two applications: medical device engineering of a vacuum-assisted biopsy device and visual effects design using a physically based flame simulation.